Search for: All records

We develop an analytical framework to appropriately model and adequately analyze A/B tests in presence of nonparametric nonstationarities in the targeted business metrics. A/B tests, also known as online randomized controlled experiments, have been used at scale by data-driven enterprises to guide decisions and test innovative ideas to improve core business metrics. Meanwhile, nonstationarities, such as the time-of-day effect and the day-of-week effect, can often arise nonparametrically in key business metrics involving purchases, revenue, conversions, customer experiences, and so on. First, we develop a generic nonparametric stochastic model to capture nonstationarities in A/B test experiments, where each sample represents a visit or action associated with a time label. We build a practically relevant limiting regime to facilitate analyzing large-sample estimator performances under nonparametric nonstationarities. Second, we show that ignoring or inadequately addressing nonstationarities can cause standard A/B test estimators to have suboptimal variance and nonvanishing bias, therefore leading to loss of statistical efficiency and accuracy. We provide a new estimator that views time as a continuous strata and performs poststratification with a data-dependent number of stratification levels. Without making parametric assumptions, we prove a central limit theorem for the proposed estimator and show that the estimator attains the best achievable asymptotic variance and is asymptotically unbiased. Third, we propose a time-grouped randomization that is designed to balance treatment and control assignments at granular time scales. We show that when the time-grouped randomization is integrated to standard experimental designs to generate experiment data, simple A/B test estimators can achieve asymptotically optimal variance. A brief account of numerical experiments are conducted to illustrate the analysis. This paper was accepted by Baris Ata, stochastic models and simulation. Supplemental Material: The online appendices and data files are available at https://doi.org/10.1287/mnsc.2022.01205 . 

Two-dimensional heterostructures composed of layers with slightly different lattice vectors exhibit new periodic structure known as moiré lattices, which, in turn, can support novel correlated and topological phenomena. Moreover, moiré superstructures can emerge from multiple misaligned moiré lattices or inhomogeneous strain distributions, offering additional degrees of freedom in tailoring electronic structure. High-resolution imaging of the moiré lattices and superstructures is critical for understanding the emerging physics. Here, we report the imaging of moiré lattices and superstructures in graphene-based samples under ambient conditions using an ultrahigh-resolution implementation of scanning microwave impedance microscopy. Although the probe tip has a gross radius of ~100 nm, spatial resolution better than 5 nm is achieved, which allows direct visualization of the structural details in moiré lattices and the composite super-moiré. We also demonstrate artificial synthesis of novel superstructures, including the Kagome moiré arising from the interplay between different layers.